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PREFACE 



This collection of four papers Is essentially a companion volume to an earlier 
publicallon entitled "Meaningful Measuremem In the Classroom Using the Rasch 
Model: Some Exemplars", published by ERU last year. This continuity in thinking and 
research efforts Is Important if we are to take full advantage of a possible paradigm 
shift in the way we look at and measure socio psychoiogicat processes given the 
availability of more sophisticated statistical techniques. The term '*meaningfur* as ap- 
plied to the measurement process Is interpreted by K C Cheung against a 
constructlvist perspective. 

The process of human problem solving Is a fascinating field* and as an area of re- 
search it has gone through a period of rapid expansion. In this context. I think this col- 
lection of four papers is a valuable contribullon to the thinking on problem solving and 
the measurement of it. 

In Chapter 1 , K C Cheung using the metaphor of a competence ladder to repre- 
sent the problem-solving continuum with progressive qualitative bands marking the 
steps (of a ladder) explains the theme of this volume, namely meaningful assessment 
of problem-solving activities. In Chapter 2. K C Cheung and Mooi Lee Choo explore 
different ways of assessing problem-solving activities In the classroom and at the 
same time describe a conceptual framework for problem-solving processes. Chapter 
3 (by Loh We Fong and K C Cheung) outlines the critical steps lower secondary pupils 
experienced In solving algebra .vord problems, and explains how these problem- 
solving activities could be assessed meaningfully, i.e. basing qualitative interpretation 
on quantitative measurement. Finally, in Chapter 4. K C Cheung and Loh We Fong 
analyse both qualitatively and quantitatively the main types of errors made by pupils at 
different performance levels of the problem-solving proficiency continuum. 

While this collection ol papers focuses on ditferent aspects of human problem- 
solving, it is also about psychomotrics. I think much of the methodology In this vol- 
ume was conceived through insights gained from more recent psychometric ap- 
proaches, particularly those of latent trait analysis. 

I would like to thank K C Cheung and two of our MEd colleagues. Mooi Lee Choo 
and Loh We Fong, for sharing with us their research findings and reflections on the 
measurement of problem solving processes. 



Ho Wah Kam 
Ag Dean 

2 ) April 1 991 School of Education 
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cliabing up th« Coap^tmott Laddcrt Bs>mm Thoughts on Meaningful 
A00«00m«nt ot Froblra-Bolving Tasks in tha Classrooa 



X C CBEUMa 



Introduot J on 

Thia ia tha aacond monograph in a aeries, following the publication 
of the first one on **Meaningful Neaaurementa in the Classroom Using 
the Rasch Model: Some Exemplars'* (Cheung et al., 1990). It 
endeavours to resolve issues on the assessment of problem-solving 
tasks. It demonstrates how psychometricians can assess pupils' 
progress on problem-solving tasks meaningfully. Hopefully, 
continued efforta in this direction will inform us of a viable 
pedagogy on problem*solving activities that are so highly endorsed 
in the school curriculum nowadays. 



What ia meaningful aaaaBBmant of problem-aolving taaka? 

In the first monograph, Cheung (1990) remarked that "it ia this 
view of measurement and testing within a constructivist philosophy 
that renders the measurement process a meaningful one" (p*2), and 
"it is the progression of the lower forms of knowing to higher 
forms of knowing that should be considered to be modelled on a 
continuum for quantitative measurement with qualitative 
interpretations" (P*4). This unidimensional continuum, which 
represents progressive development of forms of knowing that are 
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firmly rooted in the educational objectives of the ourr ,culu», 
should best be regarded as "resembling a pigtail fash.^r^:,ed as a 
ba«boo stem" (p. 4). Three examples from four authors were given: 
(1) -the •'part-whole" concept of fraction; (2) levels of Chinese 
language abilities progressing from "knowledge of characters", 
"grammatical sensitivity", to "reading comprehenjion" ; and 
(3) levels of state and trait computer progr .ming anxieties on 
"confidence", "errors", and "significant others". 

It should be emphasised that latent trait theories do not in 
principle render the measurement process a meaningful one. It 
serves essentially as a means to structure pupil responses so that 
objective, quantitative measurement with qualitative 
interpretations that are specific to a target population becomes 
a reality. The continuum, once established, is meaningful because 
the progressive forms of knowing can be understood in terms of 
constructivist perspectives. It is against this backdrop that the 
present monograph is cast on the meaningful .assessment of 
problem-solving activities. 

Assessment of problem-solving activities is not easy because 
problem solving is a very complex form of human behaviour. Pupils 
need to recognise the starting points, understand the finishing 
points, search for alternative solution paths, control and execute 
the planned steps of the problem tasks. This process is not unique 
to individual pupils unless the problems are routine, exercise-type 
questions. Research on problem solving shows that experts and 
novices differ in the deployment of perceptual cu^s, recognition 
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of a •'deep structure" of the problem, equipment of a 
domain-epecif ic knowledge base, understanding of conceptual 
knowledge, proficiency of procedural skills, deployment of 
heuristic strategies, awareness of the problem-solving process and 
its monitoring « 

In order to understand this process and its pedagogy, a theory 
of problem-solving is needed. This theory needs to account for how 
problem-perception influences problem-solving approaches, the 
problem-solving stages/cycle, how information is processed within 
our cognising mind, and alternative conceptions within the 
domain-specific knowledge base of the problem tasks. Meaningful 
assessment of problem-solving tasks is an attempt to provide for 
each pupil a quantitative measure of problem-solving ability with 
desirable measurement properties such as linearity and objectivity, 
together with qualitative interpretations that are firmly rooted 
in an adequate theory of problem-solving • Some pertinent conceptual 
and methodological issues identified can be resolved. T!ie ensuing 
chapters are attempts to illustrate these concepts of meaningful 
measurement using one example on pupils' solving of algebraic word 
problems. 



What problems are we facing? 

We can start with simple problems facing teachers everyday. How 
can we sc?t a classroom test, for example, on the solving of algebra 
word problems? How can we mark these questions? How can we arrive 
at a quantitative measure of ability in solving algebra word 
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problems? Can we know from this ability measure the progress of 
problem solving and the most probable types of errors made by the 
pupils? How can we assess pupils' work if there are a variety of 
solution paths? Can pupils be more conscious of their problem- 
solving processes? These questions, if answered, are valuable in 
informing us of a viable pedagogy on solving algebra word probiems, 
in the light of an adequate theory of problem-solving mentioned 
earlier* 

How to Bolym thmsm problems? 

Problem 1: Explicating irne "deep structures" of problem tasks 
Teachers need a blue-print in setting an open-ended test* This 
blue-print, which conveys ^;he "deep structures" of the questions 
in the test, forms the ba.«;is of assessment* The concept of "deep 
structure" is a tricky cne because even researchers on problem 
solving can have different opinions on an adequate definition for 
test construction purpof.es. In,j;^vi.3 mono graph ^ questions ore sajd 
to_gogses 3 the s am e "de ep structure" iC they_catrL . be solved by the 
pupils in partic ular »avs as inte nded bv__the teachers*, _Qn^e-the 
rtin^nt; perceptMaJl 'PMes thaLt^are_bull t-_in^to_the_qu^ pns _are 

recoqn i sed , A 1 so . agsociated with each "deep structure" is 

«?ojna tiir^.ejeiiiP Jy^oKLe^ t_t;iie _ab il i ty. . .1 eye 1_ of 

the p up 4,ls unde r exa ^^i inatio n* Thus, the same problem may be solved 
m different ways by pupils of different grades/abilities because 
the "deep structures" of the same problem are different for the 
pupils* 
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This feature of subjectivity on an explication of the "deep 
structures** of problems concurs with the test grids/plans of 
multiple choice questions. Whixe the multiple choice test is 
regarded as objective because of its pre-defined scoring 
procedures, the art of test construction in terms of contents and 
emphases is highly subjective. Realising this analogy, what is 
required in the assessment of problem solving is a method of 
presenting the **deep structures" of the questions as intended by 
the teachers as faithfully as possible and setting the questions 
accordingly. The ••deep structures** should indicate how the 
questions nay be solved by the pupils in particular ways, deploying 
the domain-specific knowledge base as intended and implemented by 
the curriculum. The use of problem*solving networks, presented in 
Chapter 2, is a significant attempt to achieve this objective. 
Problem-solving networks allow us to assess the construct validity 
of the problems by showing that each of these problems possesses 
the designed underlying **deep structure". When all the questions 
in the test are constructed according to a common *'deep structure**, 
the possibility of aligning the problems onto a unidimensional 
problem-solving proficiency continuum in the form of a pigtail 
fashioned as a bamboo stem is substantially enhanced. 

Problem 2: Grading pupil responses for quantitative measurement 

If quantitative measurement is desired, some forms of ordering 
of pupil responses to questions are necessary. There are at least 
two ways, each of which is applied to different purposes of 
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measurement. The first one resembles the impressionistic marking 
of compositions. Based on some performance criteria, ordered grades 
are awarded for each question of the test. Across the questions, 
the range of grades may be different. Neither would the same 
literal grades which are awarded to two different questions 
correspond necessarily to the same ability level. What is necessary 
is thbt each the ordered, graded responses should correspond to 
an unambiguous, ordinal set of competencies judged by the teachers 
as best as they ci:n» This scoring scheme is very versatile because 
only the quality of outcomes is ordered, without bothering about 
the solution paths taken by the problem solvers. This scoring 
scheme is best suited for the non-routine problems* The next step 
after scoring the items is to check the unidimensionality of the 
questioned spanning the test and to transform the ordered, graded 
responses into measures on an interval scale. 

The second method of scoring, which applies to open-ended 
questions of the routine exercise-type, requires special 
arrangements for test construction. The purpose of measurement is 
not only to provide a quantitative measure of problem-solving 
ability, but also to examine how the problem-solving steps have 
been taken by pupils of different levels of problem-solving 
ability. The problem-solving networks, representing the **deep 
structures^* of questions in the test, are needed to decide on the 
number of key steps in completing the problems. These key steps, 
deploying conceptual and procedural knowledge, may follow those of 
a problem-solving cycle such as on problem-understanding, 
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problem-representation, problem-execution, and problem-control • 
Through an examination of pupil responses, it is also possible to 
know the key steps that have been taken by the pupils. 

ThsT rate-determining steps of a question can be anywhere along 
the solution path. Should the most difficult step be problem- 
understanding or problem-reprcsentatjcn, pupils should find it 
relatively straightforward to proceed to completion once this step 
is secured* Otherwise, pupils may be blocked and ar*^ denied the 
opportunity to complete the ensuing stepr . If this is the case, the 
ability levels of the problem-solving steps following t*ie 
rate-determining steps may not be estimated accurately. Thus, after 
a set of core questions of the same "deep structure** have been 
constructed, it is important to include additional questions 
targeting at the components of the core questions. In this way, 
conceptual and procedural knowledge, which are needed to solve the 
core questions, can also be assessed accurately. The grades awarded 
to each ot the questions in the final test are then the number of 
steps completed. Although the steps differ in level of difficulty, 
the problem-solving sequence of routine questions ensures that the 
more steps a pupil completes, the higher his/her problem-solving 
ability. 

Problem 3: Aligning ordered graded responses onto a unidiroensional 
continuum 

Partial Credit modelling, an extension of the Rasch model, is 
ideal for analysing the two types of ordered, graded responses just 
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described. Chapter 3 shows how this measurement model cnn be used 
to analyse pupil responses on the Algebra test in Appendix 1* The 
outcome is an alignment of all the steps of the questions in the 
AlgeEra test onto a unidimeni»ional problem-solving proficiency 
continuum* Again, model assumptions and requirements have to be met 
before one can make use of the calibration results* Banding of this 
continuum into progressive forms of problem-solving competencies, 
and conceptual and procedural forms of knowing is also possible* 
This continuum can be likened to "a pigtail fashioned as a bamboo 
stem", showing how pupils progress towards mastery of solving of 
algebra word problems * Ideas of meaningful measurement discussed 
in the first monograph apply in full to this constructed continuum* 

Problem 4: Dimensional and hierarchical structuring of errors and 
misconceptions 

Pupils of different !«bilities commit different types of errors. 
Very often, errors committed by the high-ability pupils may not be 
observed in the low-ability pupils* This is because the low- 
ability pupils may not even understand or represent the problem at 
all* Errors can be loosely regarded as **inabilities" of the pupils, 
although the vast literature on "misconceptions" and "alternative 
conceptions" points to other perspectives. if meaningful 
measurement is to be pursued, the issue of how these "inabilities" 
are structured can inform us of the conceptual and procedural 
barriers towards mastery of problem solving* 

Errors made by the pupils cannot normally be regarded as 
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unidimensional , Nevertheless, some of these may be hierarchical in 
nature. Latent trait theories cannot be applied to mouel the 
s'.ructure of errors. Instead^, key types of errors of the questions 
in Che test can be tabulated against the band levels of the 
problem-solving proficiency continuum, showing the types of errors 
made by pupils of different levels of problem-solving ability. This 
contingency table may be explored for any key structural dimensions 
accounting for the observed frequency patterns. Dual Scaling, 
presented in Chapter 4 of this monograph, is a multidimensional 
modelling tool to unveil the hidden structure of a contingency 
table. The structural dl nsions may then be examined to see how 
the ordered bands of problem-solving proficiency are aligned with 
the types of errors. If this is carefully carried out, the first 
structural dimension will account for most of the information in 
the table and indicate the qualitative shift of the types of 
errors, including qui^stion omissions, made by pupils along a 
problem-solving ability continuum. 



The competence ladder 

The discussions lead to a very important concept on the meaningful 
assessment of problem-solving activities : The Competence Ladder, 

OnjtJhe,.left^m_oJL„Wie_^ 

' — «hiPl3 — represents the KoaressioQ^oI^jcpnce^tual and 

P ropedluna JJknswlng aiong^thgj .p tgfe I ejn-so 1 vijig_.a b 1 11 ty_Qpntijiu.um_.jrhe 

RCfigr.^ssiv.s^maj.U.fttLij^j23ne?s^^^ £Qsitip.n8 

of^tbe^^ujigsLj?! ^tiie^ I^Oder.^^ 
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arw of the proficiency continuum to the right arm of the key 
structural diwftnglonfB) of errors^ From this cpmpetence ladder^ ve 
knov not only how puplle of dif^eye^ t abilities perform, but a jjgp 
what^ typ es of errors they make > informov^ by an adequate theory of 
problem-solving, a pedagogy of problem-solving may be proposed* 
Knowledge of metacognltlve decision-making processes and affective 
behaviours of pupils during problem solving help pupils monitor 
their problem-solving processes* It Is this whole package of Ideas, 
not the modelling methodologies, that constitutes the theme of this 
monograph: meaningful assessment of problem-solving activities. 
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CHAPTER a 

A Bans or a Boon? t Kaaningful Aaaasssaiit of problM-Solvin? 
Aotivitiaa in tha claaarooa 

X C CHEOHQ ft KOOI Laa Choo 

introtfuotion 

Thia chapter explorea waya of assessing problem-solving activities 
in the classroom, it does not deny the usefulness of the "think 
aloud** methodologies and "protocol analysis" in understanding the 
processes of problem solving, particuljii ly in solving non-routine 
problems in novel situations. w.>iat this chapter attempts to do is 
to describe a viable theoretical framework on problem-solving 
processes, based on a theory of perception, a theory of 
information processing, a verr»ion of problem-solving cycle, and a 
consideration of domain-specific knowledge base. Network analysis, 
which is used to systematically organise pupils* responses to 
problem-solving activities, is introduced aw an alternative to 
flowcharts. It is also used as an aL\ to understanding the "deep 
structures" of the problem tasks. 

The legitimacy of this kind of qualitative analysis, in terms of 
tie scientific status of knowledge that is constructed, is 
discussed by drawing an analogy with the Repertory Grid Techniques 
in the assessment of Kelly's personal constructs. Ap such, it is 
argued that network yinalysis, set within the proposed theoretical 
framewoik on problem solving, is compatible with the current 
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interest in a const ructivist pedagogy. It is hoped that the ensuing 
discussion would serve to invoke further discussions and exchanges* 
It is anticipated that more issues need to be resolved before 
teachers can assess probleni'-solving activities in the classrooms 
meaningfully. Consequently, a boon to psychometricians may be a 
bane to the classroom teachers. 

What is proble»«>solving7 

Problem-solving activities can be ? alysed as consisting of stages: 
problem-perception or problem-categorisation, problem- 
representation or problem-reformulation, problem-control or 
problem-execution, and reasonably, problem-evaluation. These often 
sequential problem-solving stages correspond to a theory of 
perception logic, a theory of information processing, and a version 
of the problem-solving cycle. These ideas contribute to a viable 
theoretical framework towards an understanding of problem-solving 
processes. 

1. A theory of perception logic 

A neurological perspective, based on the idea of connect ionisw 
which is associated with nerve networks in the explanation of the 
logic of perception rather than the logic of thinking, is now 
emerging. The perceptual system is likened to a self-organising, 
pattern-making, and pattern-using system in that perceptual cues 
bring in past experiences. Upon reaching certain stimulation 
thresholds of the neural system, the perceptual cues help to 



12 



ERLC 




I 



reconstruct Images that have been visualised before* It is believed 
that the ''circular** movements of patterns and sequences of these 
constructed images in our brains constitute human thoughts (De 
Bono", 1990). Rehearsal of perceptual skills, often domain-embedded, 
is considered to result in Increased connectedness and permanency 
in the long-tenn memory stores of some parts of the patterned 
neural networks forming a domain-specific knowledge base. As 
such, practice and rehearsal enhance the pattern-using power of the 
perceptual system. Furthermore, boredom and perseverance can have 
directional effects on perception, in that some people are more 
ready to attend to perceptual cues and engage in sustained 
attention than others. Evidences from neurological researches are 
encouraging in supporting this connect ionist theory and logic of 
perception. 

2. A theory of Information processing 

Guided by a construct ivist pedagogy, osoorne and Witt rock »s (1985) 
generative learning model views the brain as an information- 
processing system for meaning construction. Under the premise that 
children are motivated to learn and they h.>ld responsibility for 
their own learning, selective attention to and sustained interest 
in an experience, as guided by the relvsvant memory stores and 
cognitive processes, result in selective perceptions and input of 
sensory information. Linkages are qenerated to relate these to the 
relevant aspects of information in the long-term memory and the 
generated linJs are viewed from the construct ivist perspective as 



13 



ERIC 




critical for the meanings that are constructed, only those 
recognised as viable are subsumed into the long-term memory. Later 
information retrieval is made easier since knowledge is structurr»d 
and ideas interrelated. Recent studies on metacognit ion illustrate 
the importance of pupils being conscious of their learning 
processes. Hopefully, they can monitor and control the cognitive 
and emotional systems in the construction of knowledge. 

3. A version of the problem-solving cycle 

A theory of problem solving hinges upon what constitutes a problem 
which C3n be delineated on a continuum ranging from routine 
exercic*es to non-routine problems. A non-routine problem for one 
person may only be a straightforward exercise for another, 
depending on the prior experiences of a person, and the possession 
and utilisation of a relevant domain-specific knowledge base, in 
non-routine problem solving, particularly those problems in novel 
situations, some modifications, transformations, reformulations, 
or reconstructions of given concepts, relations, and procedures may 
be requiired before viable solution paths are in sight. Furthermore, 
successful problem solvers are often equipped with a number of 
heuristic strategies, of which some are domain-specific, for the 
perception, representation, execution, control, and evaluation of 
the problem-so) ving processes. 

Earlier Gestalt theorists viewed problem solving as an interplay 
between the successive restructuring of the task environment and 
the recentering of the emphases/goals of a problem such that gaps 
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can be closed and inconsistencies settled. Insights are considered 
to play a part for some of the surprise solutions. Later 
Information processing theorists such as Newell and Simon (1972) 
focussed on how the critical features of the task environment 
structure the mental representations of the problem (problem space) 
in which heuristic strategies may be deployed for the search of 
viable solution paths (search space) . A task environment with no 
recognisable critical features indicative of a "deep structure" 
(schema) and its associated knowledge base may result in an 
ambiguous problem representation that may be based on the surface 
features only. This unclear representation may encourage a trial 
and error approach rather than the deployment of the more 
systematic heuristic strategies. For experienced problem so} vers, 
the perception and organisation of a manageable number of ••chunks*' 
of information is also an important factor in solving complex 
problems. 

With these in mind, a version of the problem-solving cycle can 
be understood as composed of the following five key processes which 
may not proceed in the listed sequence : (1) from problem- 
perception to ••problem-understanding'* ; (2) a recognition and 
explication of relevant knowledge schema in ••problem- 
representation^*; (3) the use of formalised, goal-oriented ru^'es 
or heuristic strategies in ••problem-execut ion^* ; (4) the exercise 
of meta-level decision-making processes in monitoring the solution 
paths in ••problem-control"; and (5) the formative and summative 
evaluation of the problem-solving processes in 



15 





"problem-evaluation". These five processes should best be analysed 
by network analysis. 

Whal im titttvorJc analysis? 

Network analysis, derived from systemic linguistics, seeks to 
clarify and make explicit the researcher's subjective scheme of 
descriptive categories in encoding and interpreting qual itative 
data. This is done in order to avoid over-simplified taxonomies, 
or extensive although selective quotation of the data, for a 
fuller explication of meanings. There are four useful network 
notations (Bliss, Monk and Ogborn, 1983). Subcategpries (bar) are 
mutually exclusive terms of a classification system such that these 
terms derive their meanings by the contrasts they make amongst 
themselves and by the increase in levels of delicacy as a result 
of further distinctive categorisation. Progressing towards higher 
levels of delicacy, terminals are reached which are terms without 
subcategories, on the contrary, the meanings of a term can be 
derived simultaneously from its distinct and independent facets and 
these should be co-selected or bracketed (bra) for the purpose of 
its mult i^faceted representation. This co-selection is analogous 
to the different dimensions of a contingency table. 

Repeated selections, whether applied to a bar or a bracket, 
render the possibility of a number of recursions (rec) into a part 
of the network for any addition of meanings, whether in 
combinations or in perspectives. Restricted env-ry to some pcrts of 
a network given some entry conditions is done through the use of 



by a teviTse bt'ioket, whi<:h poinLs to a specific term in the 
network, thus denying other combiiint i ons of entry conditions. A 
paradigm/ represented by a triice of a path through the network, is 
a legitimate combination of terms in a network so that the meanings 
derived are congruent with the intentions of the researcher and 
instances of such can be found in the data. 

irhat 1* * problem-solving nctvork? 

Research on problem solving generally reveals that experts and 
novices begin their problem-representation with specif iably 
different problem types/categories that they are able to perceive 
or recognise although experts and novices are cued differently by 
the surface features of a problem. Novices, as well as exports when 
they encounter familiar probl.?ms, search for familiar features of 
some typical problems they have encountered so that similar problem 
solving procedures can be applied. However, when faced with a novel 
situation, experts can initially recognise configurations of 
surface featu»-es and as a result of their experience suggest 
pr i nc ipl es/r.chema and make use oi anr^ociatod knowledge base to 

start categorising problems. On tlie otner hand, novices ate found 
to bar»e their cat egor i Bat i oti esf;entially on he surface ot literal 
features of a problem task. Hesear'-h also y^hows that cuippletion 
of the problem- representation or pi ob 1 em- 1 eformu I at i on, which 
resu 1 1 9- in an at t icu I at i on of a "do«»p r,t: met ut e" ot" the pr obi em, 
depen(is on the problem f^ulvf^r'f? donia i -j^pec i f ic knowle<ige 
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associated with the problem categorier;. The f^dme "deep structure** 
can correspond to a set of superficially disparate problems with 
diverse surface/literal features. Experts are at an advantage in 
making use of the knowledge base associated with the **deep 
structure" on embellishing or informing how a problem should be 
represented or reformulated. 

As a result of these different ways of articulation, there may 
be a number of solution paths to be spelled out in full in terms 
of problem execution and control. These routes define the problem 
and search spaces so that the problem solvers' actions can be 
mapped by a network, which may be documented by the protocols. The 
network depicts how information is processed and strategies are 
sequenced and deployed by the problem solvers. It is worth 
mentioning that the problem space of each problem solver during the 
problem-solving process is by no means static. Metacognit ive and 
attitudinal aspects of problem solving will exercise overall 
control over the ever-evolving search space as a result of the 
reconstruction of the problem space for the more viable heuristic 
strategies. From this perspective, a problem-solving network is 
more flexible to represent the problem-solving processes of pupils 
in a classroom than a flowchart which may not be equally applicable 
to all problem solvers especially on non-routine problems. 

The task approach, task complexity and the accompanying plans and 
actions in a specific task context for each stage in problem 
solving can be vie.ed as of lower "ranks" (ie. units of response 
data being further broken down into smaller parts) in a network 
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analysis than the probl'^m-solving stages which are, for the sake 
of network analysis, viewed as of the same ••rank*'. This "rank" 
scale determines wnat chunks of qualitative response data are 
associated in a temporal/causal sequence, at what levels and to 
what degree of delicacy, so as to represent the problem and search 
spaces of problem-solving activities adequately* 

An exasple ot a problem-solving natvork 

The following presents a problem-solving network of an algorithmic 
thinking exercise of routine problems in a course on data 
structures (binary tree) at the Junior College (grade 12) level in 
Singapore* The two example problems listed below possess the same 
••deep structure** in the relevant content area and the solution of 
which is algorithm-based* However, the surface features are 
different because the problems are framed in different contexts* 
Pupils are expected to deploy the algorithms/concepts that have 
been taught in the lessons* Since the problems are routine 
exercises, not much variations are revealed in the pupils* 
responses on problem-categorisation, problem-representation, 
problem-execution and problem-control* As such, the problem and 
search spaces are uniform across pupils and problems, and these 
spaces correspond to those of successful problem solvers* Should 
the problems be non-routine problems in novel situations, a variety 
of problem representations and use of heuristics proliferate* The 
task of representing these alternatives and detours in a single 
network becomes gigantic although not impossible* 
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1. Problem One: 

The results of a knockout football competition for schools can be 
arranged as follows: 
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The value at each terminal node in the tree is t^e name of a school 
(A,B, ...,H). The value at ea-^h other node is the goal difference 
when the "home** team (left-hand) played with the "away" team 
(right-hand). A negative value means th€i "away" team won. Describe 
an algorithm to fxnd the winning team, 

2. Problem Two: 

In Morse code, each letter of the alphabet is assigned a unique 

combination of dots and dashes. For example, the letters A, B, C, 

and D are coded as follows: 

A : 

B : 

C : 

D : 

This coding system can be represented by a binary tree such that, 
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except for the root node, each node of the binary tree corresponds 
to a letter of the alphabet* The Morse code of dots and dashes can 
then be represented respectively by traversing, starting from the 
root node, the left and right branches in the successive levels of 
the binary tree. Assuming the complete binary tree for the Morse 
code exists, UHScribe an algorithm which uses this tree to print 
the letter represented by a given Horse code. 

Figure 2.1 presents a viable problem-solving network after 
examining pupils* responses on the two example problems. Associated 
with each problem-*80lving process is a set of problem-solving 
stages, heuristic s 4.'ategies, procedural skills, and the 
domain-Fpecif ic knowledge base. In order to help the reader to 
understand the network which represents the **deep structure** 
underlying these two example problems, the following brief notes 
on binary trees is provided. 

••Trees** in computer science grow upside down. The data in a tree 
are contained in its ••nodes", or branching points, and are 
hierarchically arranged by levels. The node at the highest level 
of the hierarchy is called the ••root" of the tree, whereas the 
nodes at the lowest extremities are called the "terminal nodes" or 
••leaves^^. A ••binary tree** is one in which each node may have at 
most two subtrees: the right and left subtrees. These subtrees are 
binary trees in their own right. The usual way of representing a 
tree in computer science involves the use of "pointers", pointing 
to the addresses of arrays where the roots of the subtrees are 
stored. For a binary tree, each node consists of the data value. 
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which may be a number or a literal, plus two pointers* One or both 
of the pointers may have "null values" if they have no subtrees to 
point to* 

K tree may be "traversed" in several ways. Traversing a tree is 
accessing its stored content in a systematic way. How the tree is 
traversed is an essential part of the development of an algorithm 
in solving the programming problem and this cannot be performed 
without considering the data which the algorithm manipulates. For 
brevity, tree traversal may proceed by "level" and/or "order" such 
that a systematic sequence to reach the goal is achieved by 
combinations of "top-down" and "left-right" searches. In the data 
structure literature, methods of tree traversals such as 
"preorder", "inorder" and "postorder" constitute part of the 
domain-specif ic knowledge base of pupils faced with problems on 
binary tree structures. 

It should be noted that the two example problems differ in the 
surface features underlying the bin^ary tree data structure, in 
problem one, a positive data value indicates that the "home" team 
won and hence taking the left branch of the tree would lead to the 
winning team. Applying this scheme to the given binary tree, the 
.node of tree traversal is: 

Starting at the root, 

If the data value is positive, take the left branch, else 
if the data value is negative, take the right branch. 

At the last level of the tree, the node reached will give 
the name of the winning team. 
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For problpm two, a dot indicates taking t)ie left branch while « 
dash mnanB taking the riqht branch of the binary tree which 
represents the system of Morse codes. By following the given 
secfuence of dots and dashes, the code character can be arrived at. 
The mode of tree traversal is therefore: 
Starting at the root, 

If the next symbol is a dot, take the left branch, else 
if the next symbol is a dash, take the right > nch. 
At the end of the sequence of symbols, the node reached will 
give the corresponding code character. 
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Im network Analysis sciMtifio? 

This section attempts to draw an analogy with networks (cf . network 
analysis) and Kelly's personal constiucts (cf. Repertory Grid 
AnaTysis) so as to assess the scientific status of problem-solving 
networks* 

Kelly (1955) commented that man makes sense of the world by 
constructing and fitting representational models of reality. These 
mental models will only be amended if they are found to be wantingr 
for instance, in order to account for anomalies. As a result of 
daily experiences, man builds up his system of personal constructs 
which is essentially a system of subjective evaluation of his 
encounters against his anticipation of events* The repertory grid 
technique, Invented by Kelly, enables the elicitation and 
exploration of part of this system by simultaneously noting 
likenesses and contrasting differences of events. The basic 
philosophy is that it is not possible to affirm logically something 
without at the same time denying something else. 

The technique involves presenting a person with a series of 
different stimuli so as to trigger a different construct in each 
case. This triggering process, unlike those of the stimulus- 
response theory in behaviourism, refers to how one responds to i^hat 
he perceives to be a stimulus. Hence, the probability of two 
persons perceiving events in identical ways is not large and this 
contributes to a person *s individuality. For example, by 
associating l/Dndon with Sydney and contrasting them with Tokyo the 
construct of Type of Spoken Ininquaqe can be obtained, with English 
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speaking and Non-English speaking at the two poles of this 
reference axis. In this way, a grid of a universe of stimuli/events 
and a system of personal constructs is constructed. Ratings and 
ranJcings of stimuli, and clustering and ordering of constructs are 
obtained by outcome. Thus, personal/shared meanings of events are 
determined by where we are in this/these referenced grid 
structure/s, where the process of knowing and anticipating is 
channelled • 

Since each construct is a personal, resemblance-contrast 
reference-axis contrasting two poles such that two events can be 
likened and then simultaneously contrasted with a third, this is 
reminiscent of the bar of categories of a term in a network in that 
decisions on categorisations have to be subjective but data- 
dependent. While the poles and categories are absolute entities 
once they are d**cided, the choice of interpretation of an event 
according to this set of constructs or classification system still 
depends on how one anticipates a greater possibility amongst the 
alternatives for a more adequate elaboration of meanings, whether 
for constricted certainty or broadened understanding. 

The system of constructs of a basic concept of an event is 
analogous to the simultaneous appccts of a term within the same 
bracket in a network. However, these simultaneous aspects may 
differ in levels of abstraction and one aspect may be analysed as 
being more supciordinate than the other, since a construct allows 
us to locate personal meanings and anticipate events, constructs 
are necessarily cross-referenced by th ? users. Thus, the ordered 
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nature of confitructs in a personal r>yr>tom ir. pnr:\i\fil to the levels 
of delicacy of a network whereas the cross-referenced, structured 
nature, to some extent, may indicate the branching, recursion, and 
restricted combinations of some basic terms of a network- 

Like the constructs which are useful for the anticipation of a 
finite range of events, the paradigms of a network are usually 
finite and can be instanced in the qualitative data. Furthermore, 
a personal construct system is ever-evolving in coping with 
anomalies and daily experiences. Thus, a succession of networks is 
necessary to indicate changes in conception which is a fundamental 
process of learning. This is because each of these networks 
resembles a static snapshot of the personal construct system on how 
we may perceive and anticipate events. The fallibility of the 
system of personal constructs in the light of anomalies, and the 
openness and adjustability of the system to include new constructs 
or to modify old ones, whether they are superord Inate or 
subordinate, are close to the Idea of Insufficiency of 
detall/scope/notat Ion of a network structure in coping with new 
circumstances. 

One may also view knowledge in pieces, an idea from di Sessa 
(1985) that these pieces are loonely-coupled in a flf?xlble, taelf- 
contained manner so that the personal system of constructs can be 
used more creatively. Branching within a network may indicate this 
kind of knowledge structure as well, Kelly maintained that where 
one personal construct system i£? similar to the other, the 
corresponding psychological piocpsson involved am alf^o similar. 
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Likewise, each paradigm of a network should correnoond to n fixed 
set of meanings and processes so thc«t the frequency of occurrences 
of this paradigm can be counted in the network data in an 
unambiguous manner. In this case, the network can represent shared 
meanings/perspectives of each instance of tho network data. 

In summary, network analysis, like the repertory grid technique 
and its associated Personal Construct Theory, is a way of 
organising experiences and knowledge. Since human knowledge is both 
personally constructed and socially shared, the network of 
qualitative data representing such knowledge is necessarily linked 
to a process of knowing. In this way, Kelly's ideas of personal 
constructs and the powerful paradigm of constructivism can furnish 
a viable philosophical base on how networks should be constructed 
and utilised. Criteria of validity of a network such as the 
viability of a network in coping with old and new data, can be 
examined accordingly. Without a philosophy of knowing, network 
analysis remains a useful tool in coding and representing 
experience only. 
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CHAPTER 3 

on Meaningful K«nnur«a«ntt 8t«gft« of Lover Secondary Pupile* 
Abllltlee in Solving Algebra Word Pzobleme 



LOH we rong ft K c CHSONO 

soae oonteaporary views of 4S theory of problem solving 

uower secondary pupils experience considerable aifficulty in 
solving mathematics word problems. Despite the atteipot by 
curriculum planners to include more i roblem-solving activities in 
the mathematics curriculum and the large volume of research on 
"expert-novice" problem-solving, problems rel'^^ied to how to tea-^h 
for both conceptual understanding and procedUi.A proficiency at the 
lowei secondary level appear to have persisted. Perhaps two major 
problems ore that meaningful assessment of problem-solving 
£.:;tivities is not an easy task for classroom teachers, and that a 
viable pedagogy of problem-solving in mathematics is by no means 
clear. This chapter seeks to resolve the first probl'^m of 
meaningful assessment, whereas the second issue of pedagogy will 
be explored in Chapter 4 of thi;s monograph. 

Chapters 1 and 2 of this monograph outlined a perspective of 
problem-solving, taking into account a theory of perception logJ*". 
a constructivist theory of information processing, a version of a 
problem-solving cycle, and a consideration of an associated 
domain-specific knowledge base of the problem t;»:3ks. It is clear 
tSat without such a theoretical framework, recommenciat ions on a 
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viable pedagogy on problem solving may not be meaningfully 
proposed. Furthermore, an advocation on the use of problem-solving 
networks in constructing problem-solving tasks and analysing 
response behaviours is a first and significant, step towards 
e\presslng what processes should have been undertaken, and 
describing what and how the outcomes are obtained when pupils are 
engaged in problem-solving activities. Network analysis, unlike 
research-based techniques such as "protocol and "episode" analyses 
used in "think aloud" researches, is recommended for its 
versatility in summarising pupils' solution steps and results, and 
clarity for classroom teachers in explicating the designed "deep 
stiuv-ture" of problem-solving tasks during test construction. 

6ome known misconceptions in solving algebra word problems 

Recent studies have shown that success in solving mathematical word 
problems is due to abilities in con-jtruct ing appropriate 
representati-jns of problem situations, reformulating problem 
conditions, developing solution plans using a relevant domain- 
specific knowledge base, and deploying procedures and heuristics 
to generate viable solution paths (Davis, " 85; De Corte and 
Versnhaffel, 1985; Venezky and Bregar, 1988). Amongst these 
abilities which teachers and pupils find troublesome to teach and 
learn are problem-understanding and probl em-reprepentat ion , which 
are critical stages in a number of proposed models of mathematical 
problpr^-sol *n { -^ch >en and Oohmke, 1980; Davis and silver, 19P'3? 
I^sh, Landau and Hamilton, 190^; Riley, Gieeno and Holler, 1983? 
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Bransford, et al., 1986; Venezky and Bregar, 1988; Kouba, 1989; 
Schroeder and Lester, 1989)* 

Put in a simple way, problem understanding helps the construction 
of an Initial representation of a problem, and the setting up of 
explicit goals which will direct the course of problem solving 
(Janvier, 1987? Lesh, et al, 1983; Sliver, 1987). In this regard, 
Resnick (et al., 1981) proposed that there are at least three 
possible ways to represent a problem: (1) informal or linguistic, 
(2) physicbl or visual, and (3) algebraic. Combinations of these 
ways may be applied in a single problem-solving task, affecting 
the heuristic strategies and metacognitive decision-making that are 
deployed during the problem-solving process (I^sh et al., 1983). 
Apart from those difficulties that arise from the problem-solving 
process, misconceptions in the content and process skills of 
algebra can hamper the successful solving of algebra word problems 
as well. Some of these misconceptions identified in a number of 
research studies are: 

1. Different interpretations of letters and variables - Children 
generally do not understand the idea of i letter as a variable. 
They tend to interpret letters as standing for specific numbers, 
and different letters must necessarUy represent different numbers. 
They do not understand that a letter can represent zero or can take 
on any value, positive or negative. Some are confused over the 
distinction between a letter as representing the number of a given 
object, and as a shorthand notation representing the actual object 
(KucjAemann, 1981; Booth, 1984). 
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2. Improper use of brackets - Children ignore the proper use of 
brackets by assuming that the problem context determines the ord#>r 
of operations, or that operations are simply performed starting 
from^left to right within an algebraic expression (Booth, 1984)* 

3. The need to give numerical or single-terra answer - Children are 
reluctant to record an algebraic statement as an answer, thinking 
that a numerical answer is required (Booth, 1984). They also tend 
to give numerical values to letters and variables at their earliest 
opportunity (Col lis, 1975; Kuchemann, 1981; Booth, 1984; Ellerton, 
1985; Dickson, 1989). 

4. Different interpretations of terminology such as "solve 
"simplify", "factorise" and "evaluate" - Children have difficulty 
in deploying the forir .1 methods that have been taught to them even 
in the case of simple arithmetic. An example is that the 
procedures used by pupils in solving arithmetic problems are 
difficult to be symbolised (Booth, 1984). Terminology and the rules 
of algebra that are obvious to teachers may be a source of 
puzzlement and confusion to the children (Thwaites, 1982; Ellerton, 
1985) . 

5. Understanding of the equality symbol - Equations of the form 
"a^bx+c" are misinterpreted as trying to give a numerical answer 
to the "stim" on the right-hand side of the equality symbol . 
similarly, equations of the form "ax + b-cxtd" are scon as two sums 
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to be evaluated. The equality symbol is not conceived as an 
operation which should be applied identically to both sides of the 
equation. Father* it is viewed as an operator symbol requiring the 
respondent to do something on the "sums" that are given separately 
on both sides of the equation (Behr, Erlwanger and Nichols, 1976; 
Kieran, 1981; Dickson, 1989). 

6. The tendency to invent "mal rules" This occurs when some 
prototype rules are created by generalising taught systematic rules 
in order to extrapolate new rules which are considered by experts 
as incorrect (Radatz, 1979; Matz, 1982; Sleeman, 1982; Thwaites, 
1982; Van Lehn, 1983 ; Pesnick, et al., 1987). An example on the use 
of "malrules" is to extrapolate "a^^ (b*c) =^ (a + b) ♦ (a+c) " from the 
taught systematic rule "a* (o+c) - (a*b) + (a*c) " by creating a 
prototype rule obeying the distributive law of algebra. 

Purposa of study 

Children attend differently to contextual cues, gain differential 
access to appropriate domain-specific knowledge base, deploy a 
diversity of heuristic and metacognit ive strategies, search for 
viable alternative solution paths, commit various types of 
conceptual and procedural errors, and are found lost in detours and 
tvapped in blind alleys in their solving of algebra word problems. 
The purpose of this chapter is to examine how lower secondary 
pupils in Singapore enqaqe in solving algebra word problems, and 
to explore how problem-solvinq activities can be meaningfully 



33 



ERLC 




assessed according to the criteria described in Chapter 1 of this 
monograph. 

Network analysis of two routine exercise-type algebra word 
problems (items lo and 11 in Appendix 1) reveals the commonly 
designed "deep structure" of the problem-solving taskii. This allows 
other simpler questions (items 1 to 9) to be constructed, based 
only on some key problem-solving steps such as problem- 
understanding and problem-representation and some Key conceptual 
and procedural skills listed as components in this network. 
Analyses of the problem-solving steps taken by the pupils and 
their misconceptions contribute to a scoring scheme on item 
performance levels, which reflect critical response thresholds 
along the solution paths for successful problem solving. Through 
the use of Partial Credit Modelling, the calibrated item response 
thresholds are checked to ascertain whether they behave as designed 
within qualitative, progressive bands such that the levels of both 
conceptual and procedural understanding they represent are firmly 
rooted in the problem-solving research literature. 

Design of the sanple and algebra test 

The sample for this study consists of 130 secondary two (grade 8) 
pupils chosen from four classes of a typical government-aided 
secondary school in Singapore. The sampl-s comprises one class of 
high-ability pupils, two c.asses of average-ability pupils in the 
"express" stream, and one cla^s of below-averaqe ability pupils 
from the "normal" stream, in Singapore, the "express" stream pupils 
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normally take four years whereas the "normal** stream pupils take 
at lea&t one additional year to complete their secondary school 
education* 

Algebra is first introduced to pupils in Singapore when they are 
in primary six. At the end of secondary one, pupils are expected 
to be able to simplify algebraic expressions of one unknown which 
may involve brackets, evaluate algebraic expressions by 
substitution, use symbols and letters to represent numbers and 
express physical situations, solve simple algebraic equations and 
algebra word problems (Ministry of Education, Singapore, 1981). 
Appendix 1 contains 11 questions, with items 7 and 9 In two parts, 
pertaining to this syllabus* since the purpose of this test is to 
establish a progressive, qualitative continuum indicating how 
competencies may be deployed and barriers may be surmounted, those 
conceptual knowledge and procedural skills leading to the 
successful solving of items 10 and 11 ate built into items 1 to 9 
of the test. Otherwise, should problem understanding and problem 
representation be the most difficult &^eps, the difficulties of the 
ensuing steps on problem execution and control may not be estimated 
accurately* This is tecause pupils are not given the opportunities 
to demonstrate their competencies when they fail to secure a good 
start* 

The common designed "deep structure" of items 10 and 11 is shown 



in Figure 3*1* Based on this problem-solving network and 
alternative conceptiofT obtained from analyses of pupils* 
responses, the performance score levels of each of the test items 



35 




ERIC 



....... / V........ . 



T r 1*1 Ai^rl - 



~ At •« t * t> % « » t crt 

• III mr- k at a 

— II 1 t I 'V t 

tii.hrit. iwi. t m I 
^««toit t^rlAtola to iii«»> i«i->w«* 



36 



42 



are determined. These levels are sliown in Table 3»2, vhere their 
calibration results are also presented and discussed later. It 
should be noted that the test being of routine exercise-type 
enables ^precise definitions of item performance score levels 
although it is observed that some pupils use trial and error 
methods when they fail to represent questions 10 and IX for an 
algebraic solution. This issue of quantitative modelling of 
alternative solution routes, an educational more than a statistical 
problem for researchers, remains unresolved if a combined analysis 
is to be sought. 

The use of partial credit mc^lelling in the construction of the 
algebra problem-solving proficiency continuum 

Partial Credit Modelling (Masters, 1982) was used to analyse the 
pupils* ordered, graded responses of items according to a scoring 
scheme so that partial credits for partial success are assignable 
to items that can be mapped on a unidimonsional continuum. Partial 
Credit Modelling permits the maximum item !5cores, which reflect the 
highest performance levels attainable of pupils, to vary across 
items. In addition, it allows the structure of competencies or 
procedural stops that these performance score levels represent to 
vary across items. The proposed scoring scheme mentioned earlier 
serves precisely this purpose of grading pupils' responses into an 
ordinal scale of performance levels. Table 3.1 presents a frequency 
distribution of performance levels of test items based on pupils' 
responses that have been graded according to the proposed scoring 
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Table 2.1 Frequency Distribution of Performance 
Levels Across Items 



Item Item Performance Level (Item Score) 





1 
(0) 


2 
(1) 


3 
(2) 


4 

o: 


1 


10 


120 






2 


29 


40 


61 




1 

•J 


2 


13 


115 




4 


17 


13 


100 




5 


19 


14 


30 


67 


6 


22 


108 






7(1) 


24 


4 


102 




7(11) 


33 


6 


91 




8 


55 


8 


67 




9(1) 


73 


11 


9 


37 


9(11) 


18 


6 


106 




10 


80 


7 


35 


4 


11 


45 


6 


46 


11 
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scheme • 

The outcomes of the Partial Credit Modellinq, if the model 
assumptions and requirements are met, are sets of pupil ability and 
item step difficulty parameter estimates on a common linear logit 
scale. Table 3.2 shows the item step difficulty estimates and their 
associated fit statistics. Figure 3.2 presents the calibrated 
ability distribution of the pupils in the sample. The various item 
steps of the algebra test items demonstrate excellent statistical 
fit to the Partial Credit Model and they span a long range of at 
least six loqits (item step difficulties estimates range from -'3.27 
to 2.95) with the average difficulty of the item steps being 
arbitrarily set at zero logit. Judging from the ability 
distribution of the pupils, it can be concluded that the test has 
been appropriately targeted at the pupils in the calibrated sample 
because the modal region of the roughly normal distribution is 
approximately at 0.8 logit. All of these statistics demonstrate the 
internal validity of the algebra test. 

The item ^itep difficu/ty estimates are not necessarily ordinal 
because they are response thresholds indicating the difficulty of 
proceeding from one given performance level to the next when a 
pupil is already at a given level. Notwithstanding this, a pupil's 
ability logit estimate generally increases with increasing 
performance score levels. It is noteworthy that some intermediate 
performrjMce score levels are less likely to obtain because once the 
cognitive or procedural barriers have been surmounted the ensuing 
steps simply follow. It is this "step" property which renders 



Table 3.2 Item Bat.lm.ites and 


The 1 1 Fit. 


Stal isl ics 








DMcription Step Difficulty 
of lt«m ttvp Ettlmats 

l«8lApi fy t»rm« "2. U 


Precision 
(Standard 
error ) 
• 34 


Fit 

V 

1«00 


Statistics 
q t 

.27 .10 


2«RtAov» bracket 
Slmpll fy t«rA« 


.17 
.13 


• 23 

• 19 


• 93 


.17 


-•74 


3*8ub«titut« valuvt of «ib|C 
£v«lu«t« «)(pr«««lon 


-2,11 
'1.78 


.76 
.29 


1.21 


.26 


.82 


4*Sep«rAt« conttAnt 9( unknown 
Solve fr • y 


.27 
-1 54 


.23 
.23 


• 76 


• 16 


''1.66 


StMultiply out brACk«t» 
Add 4x^8 to 2x 
Solve for x 


.17 
-.50 
-.20 


.29 
.24 
.20 


.96 


.12 


-.30 


e.Tranilatv twlcv of x to 2x 


-1.22 


.25 


.93 


.15 


-.45 


7i)Know relatlonvhlp b«tw«tn 
variables froA word sentence 
Write algebraic expression 


2.38 
-3.27 


.26 
.25 


.73 


.15 


-2.04 


7ii)Know relationship between 
variables from word sentence 
Urite algebraic expression 


2. 12 
-2.43 


.23 
.22 


• 91 


. U 


-.81 


6. Intepret 5m and 2p 
Interpret 5m ♦ 2p 


2.30 
-1.67 


.21 
. 20 


1 . 07 


.09 


.77 


9i)Know rvlationship between 
variaoies ironi wora sipnvcn^e 
Write overtime pay as 2h 

Equate w=500 ♦ 2h 


i . jyJ 

.90 
-.51 


.21 
.23 
.24 


1 .08 


.11 


.69 


SiDCalculate overtime pay 

Calculate total wage 


1.22 
-2.32 


• 29 

.26 


.92 


.17 


-.46 


10. Write first equation 
Write second equation 
Solve equations partially 
Solve equations completely 


2.82 
-1.04 
2.95 
1.26 


.21 
.22 
.39 
.55 


1. IB 


.13 


1.34 


U. Write first equation 
Write secofid equation 
Solve equations partially 
Solve equations completely 


2.30 
-1.80 
1.98 
.30 


.22 
.22 
.24 
.27 


i.Ol 


.11 


.14 
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r^caXe Raw Ability FrequfMicy 
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Klqurc 3.2 Ability l;ln».r Ibul Ion of Pupils 
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Partial Credit Modelling to be different from tliose graded response 
models that are based on "category boundtiry" approach, of which It 
must always be easier to reach a low performance level than the 
higher ones. By considering the odds of passing from a given 
performance level to the next and equating these odds to a set of 
logistic probability curves, the Paxtial Credit model is actually 
an extension of the Rasch and Rating Scale model in the number of 
steps taken and handling of inordinal item step difficulties. 
Consequently, this measurement procedure satisfies all the 
requirements such as specific objectivity and linearity of a 
measurement model (Cheung, 1990) and meaningful measurement such 
as those exemplified by Koh and Cheung (1990) is also achievable. 

The item stop difficulty estimates of all the test items help to 
define the progressive, qualitative aspects of the constructed 
proficiency continuum on the solving of algebra word problems. 
Special care should be taken to interpret these estimates because 
they indicate the response thresholds of completing those steps 
that are associated wHh the various performance score levels, and 
not the actual abil^tv level of the known competencies that may be 
applied .thin the problem-solving sequences. Thus, the 
problem-solving sequences of the test items as explicitly 
reflected in the scoring scheme, and tJie patterns of response 
thresholds particularly those of the rate-determining steps, are 
important considerations in establishing the external validity of 
the problem-so: ving proficiency continuum. 

This cont ent analysis is summarised in Figure 3 . 3 as an item map. 
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indicating the probability of more than 0.5 of scoring the 
performance levels 1 and above rather t^an 0, 2 and above rather 
than 0 and 1, and so on across all items that have been arranged 
in an'^order along the proficiency continui.m. This continuum is 
found to match with the design properties of the test. The actual 
probabilities are not shown in this item map. Instead, separate 
••zone" maps of each item can be drawn, indicating the probability 
of scoring at the various performance levels. Figure 3.4 shows an 
example zone map of item 8, of which the first step is the rate- 
determining step, so that the probability of scoring one is 
negligible. In passing, one may want to know that the item map can 
be obtained from the set of zone maps by drawing a line through 
••probability « 0.5^^ on each zone map to obtain the most probable 
performance levels along the ability logit continuum. 

Quantitative measurement with qualitative interpretations 
using the proficiency continuum 

Items 1, 3 and 6 help to define the lowest portion (scale 
performance level « 1, up to -0.9 logits) of the continuum. They 
involve simplifying •'2a + 5b+a" to ••3af5b'» for item 1, substituting 
the values of a, b, and c into "3a-b*5c«« and evaluating this 
expression correctly for item 3, and translating word sentences 
into algebraic expression ••2x»» to indicate "twice of for item 
6. Hin response thresholds of these steps in logits are -2.11, 
2.11 and -1.78, and -1.22 respectively for the three items. 

Items 4, 7(i) and (ii) , and 9(ii) contribute to a definition of 
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the next lower portion (scale performance level « 2, -0.9 to 0.3 
logits) of the continuum. Amongst these three items in four parta, 
the first steps are all critical (response thresholds are 0.22, 
2.38, 27x2 and 1.22 respectively) and the ensuing steps simply 
follow (-1.54, -3.27, -2.49 and -2,52 respectively). These critical 
steps include separating the constant and unknown terms of 
••4y-8-2y-4" into "2y=^4*' for item 4, understanding the two algebraic 
relationships of the )cnown and unknown from word sentences for 
items 7(i) and 7(ii}, and forming the algebraic equation 
"w«500f2*10" for item 9(ii). The ensuing steps respectively involve 
solving for "2y^4'\ coding correctly expressions "20-x" and "x-y", 
and calculating the values of "500+2*10". With hindsight, one can 
see the flexibility and power of Partial Credit Modelling in 
treating what should constitute a problem-solving step. For 
example, one can easily regard the ensuing steps as part of the 
package of critical steps. Thus, these ensuing steps are not 
explicitly estim^ttnd without any major consequential effects on 
the overall qualitative structure of the proficiency continuum. 

Items 2, 5 and 8 contribute to an understanding of the higher 
portion (scale performance level « 3, 0.3 to 1.5 logits) of the 
continuum. The response thresholds of the two steps of item 2 are 
very close (-0.17 and O.n) and the completion of which indicates 
a person is progressing from scale perform<^nce level 2 to 3. These 
two stepr involve grouping the like terms together by removing the 
brackets of " ( By ♦ 62) - (4y + 37.) and simplifying this expression to 
form *My*37.". v';imilarly, the response thrcsl.olds of the three steps 
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of item 5 are very close, with the first step a little bit 
relatively more difficult (0.17, -0.50, -0.20). The completion of 
these steps prepares one for entering into scale performance level 
3, The'" steps include multiplying "4(x + 2)" to form ••4x + 8", then 
adding "4x+B" and "2x" to form "ex+S" and finally solving "6x48-4" 
to give "x«-2/3". Item 8 is a two--step item with a difficult first 
step and easy second step (2.12 and -2.49). The first step involves 
an interpretation of "5m" and "2p" in a mathematical rather than 
informal qualitative way, and the second step an interpretation of 
the algebraic ex. ression '*5mi2p** as the cost of 5 mangoes and 2 
papayas. 

Completion of items 9{i), 10 and 11 shows the clearest 
indication that a person has attained scale performance level 4 
(greater than 1.5 logits) . The response thr<jholds of the three 
steps of item 9(i) are 1.90, 0.90, and -0.51 respectively, 
indicating that the first step is the most difficult step. This 
item requires the pupils to understand the relationship between 
total wage, basic wage, and overtime pay, and then express overtime 
pay as part of the total wage, followed by equating ••w=500 + 2h". 
The item response behe^viours of items 10 and 11 are similar, with 
each of the steps of item 10 more '^.ilficult than those of item 11 
because the latter is a standard example in the textbook. The 
response thresholds are 2.82, -1.04, 2.95 and 1.26 for itom 10, and 
2.30. -1.B2, 1.98 and 0.30 for item 11. As revealed, the first step 
of both items, which requires the pupils to translate the word 
sentences into algebraic equations, are the rate^determining steps. 
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The second step of both items, which are relatively straightforward 
once the first steps ^re completeJ, seeks to supply a second 
equation for the simultaneous equations to complete the problem 
representation. The last two steps involve substituting one 
equation into another and solving for the two unknowns accordingly* 
It should be noted that representing the problems in an algebraic 
way and solving a system of simultaneous equations in two unknowns 
are difficult tasks for the pupils. 

A point noteworthy of documentation is the attempt in this study 
to incorporate those successful problem solvers who obtained their 
answers using trial and error (approximately 251 of the sample) 
into the Pattial Credit Modelling procedures. This is done by 
grading these responses at performance level 2. While this 
practice warrants further examination in Chapter 4 and is unlikely 
to be resolved on statistical groundr, unly, the internal validity 
of this proficiency continuum appears credible because of the 
conformity of the response data with the stringent measurement 
model. It is anticipated that pupils* responses obtained from 
differetit solution paths may need to be modelled separately. They 
may be combined only if the item response behaviours are 
statistically conformaole to the Partial Credit model and the 
corresponding performance levels of the alternative solution paths 
as laid down in the scoring schemes can he e*iucat iona 1 ly justified. 

Conclusion 

This paper demonstrates the possibility of model! inq j rublom 
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solving tasks of the routine exercise-type so that quantitative 
measurement with qualitative interpretation is possible. This is 
a first but significant step towards meaningful assessment of 
proBlem-solving tasks laid down in Chapter 1 of this monograph. At 
least two other issues need to be resolved. First, the various 
solution paths may not be equated for the assignment of performance 
score levels because of the different structures of competencies 
involved and uses of heuristic strategies during the 
problem-solving process. It is also because of the varying degrees 
of opportunity to learn of the pupils. Second, there are 
metaco-jnitive and attitudinal aspects of problem solving informing 
how errors, false starts, detours, and blind alleys have come 
nbout. These information, if integrated with the problem-solving 
proficiency continuum, makes meaningful measurement of 
problem-solving activities more complete. Some of these are 
examined in Chaptei- 4. The findings on the difficulties of solving 
algebra word problems in this chapter not only substantiate those 
in the literature, but also statistically demonstrate that 
problem-understanding and problem-representation are critical 
rate-determining steps in problem solving, even if the problems 
are of the routine-exercise type. 
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CHAPTER 4 

on HMningful M«a«ur«a«nti M«taoogtiltion and Kiararchlcal Modelling 
of Brrora in Algabra Word Problaas 



K C CHEUNQ ft U>H Wa Fong 

8oma oontamporary viava of aatacognition on mathanatioal problaa 
aolving 

when a person is actively engaqed in an attempt to solve problems, 
particularly non-routine problems, there are meta-level 
decision-makinq processes which are most likely to occur between 
transitional episodes of the probl rm-sol v inq process, indicati;iq 
the occurrences of reflective, reijulatory, evaluative and control 
behaviours (Schoenfeld, 1983). Biqqs (1987) summariood six kinds 
of metacoqnit ive knowledge: (I) content-knowledge of what one Is 
talkinq ahovit; {2) se 1 f -knowl edqe ot what one already knows or does 
not know; (3) for o-k»mwledqe of purpones and qoals; (4) situational 
awareness of the w.^ys thinqs are; (5) ptoqiess-knowledqe of where 
one is qoing or chanqing course; and (6) st rateqic-knowledqe of 
alteri^ative ways to work best in circumstances. 

Kilpatrick (1967) pioneered tl^e use of protocol coding schemes 
to examine coqnltive processes. Since then, the use of verbal 
reports r.ijch as those obtained from the "think-aloud" procedures 
provides legitimate data for analysinq probl em-sol v I nq >)ehaviours 
(F'.ricnson and Simon, 1980). Patterns of behaviours and stieams of 
events of thone onqoinq coqnitive processes that trome to conscious 
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attention, aa opposed to those obtained through introspection and 
retrospection, are analysed into sequences and episodes for 
explan<ition through processes of metacognltion and affection. From 
an information theorist's standpoint, it is essentially those 
psychological data in the short-term memory, and the 
problem-solving processes carried out in the central processor that 
are the focus of the study. 

Polya's famous book **How to solve it 7" provides a general 
four-ph<-\se model consisting of heuristics "Examine-Plan-Do-Check" 
of the problem-solving process on mathematical proHloms in a 
variety of contexts. Recently, Schoenfeld (1987) devexoped four 
within-classroom t<»chniques for the teaching of heuristic 
Rtr«itogles with a mptacogni t i ve focus: (1) using videotapes to 
create awareness of the thinking and metacoqniti ve processes, (2) 
teacher as a role model for metacogni t ive behaviour, (3) teacher 
AS a moderator rather than an arbitrator in whole class problem 
solving, and <4) teacher as a facilitator in small group problem 
solving. Dcfipite all these efforts, a viable full-fledged podagogy 
of problem solving is not available mainly because a theory of 
problem solving is not fully developed yet. Chapter 2 oi this 
monograph is an attempt to provide some ideas for a more adequate 
theory. It is hoped that the findings of researches and teaching 
experiments can inform us of a better piviaqogy for use in the 
classroom. 
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Furpoat of study 

This study, a continuation of Chapter 3 of this monograph, seeks 
to analyse toth qualitatively and quantitatively the key types of 
errors made by the pupils at the different performance levels of 
the problem-solving proficiency continuum. For the purpose of this 
study, five items along this proficiency continuum are selected* 
They represent a list of concepts, procedural skills, and key types 
of errors on solving algebra word problems. The key types of errors 
and the bands of performance levels of solving alqebra word 
problems are then modelled using dual scaling, a multidimensional 
scaling procedure, to ascertain the dimensional and hierarchical 
nature of errors. Metacognitive behaviours are examined to 
explicate the problem and search spaces of the problem-solving 
processes. Guidelines for a pedagogy of solving algebra word 
problem are also presented* 

Error analysis of selected algebra irord problems 

Error analyses of five test questions (3, 4, 5, 8 and 10) along the 
algebra problem-solving continuum are reported here althou'jh a 
full-scale analysis of the whole test can be done as well. Most of 
the errors on these five items are made by pupils with scale 
performance levels 2 and 3 (87% of the sample) . However, there are 
some key types of errors that are mado more often by pupils at one 
level than the others. At the same time, some types of errors are 
committed by pupils at the four levels of performance. For the 
purpose of this study, key types of errors aie those errors viewed 
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by the present authors ae edu< <\t ional iy b Iqni f ic:<*nt and ar© 

committed by a large number of pupils in tho sample. Table 4.1 

shows how the key types of errors are distributed across the four 

performance levels of the problem-solving continuum. 

Item 3 (sc^le performance level 1): 

What is the value of 3a-b+5c if a^2, h^-l and c*5? 

Nearly 90% of the pupils answer this question correctly (601, 
91%, 90%, 91% respectively at the four levels of performance). The 
key error is that pupils at levels 2 and 3 (8% and 10% of pupils 
at the two levels respectively) evaluate the numerical value? 
wrongly after substituting the values of a, b and c into the 
expression correctly. The response thresholds of the two steps of 
substitution and evaluation are -2,11 and -1.78 logits resF^ect i vely 
and this key error occurs in the slightly more difficult second 
step of evaluation. 

Item 4 (scale performance level between 1 and 2) : 
Given that 4y-8^2y-4, find the value of y? 

Nearly 80% of the pupils answer this question correctly <0%, 
64%, 93%, 100% respectively at the four levels of performance). The 
key error is committed at the second step where pupils at 
performance levels 2 and 3 (16% and 6% of pupils at the two levels 
respectively) are required to simplify the constant and unknown 
terms. This is done after the first stop of grouping the terms to 
elthor sidos of the equality symbol in ordor to solve the equation. 
The rerponso thresholds of the first step ol leaiianqinq the 
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terms, and the secona step of simpl ifying and solving the equation 
are 0.22 and -1.78 logits respectively. 

Compared with the second step of item 3, the second step of item 
4 invited^more pupils at level 2 to commit errors although the step 
difficulties are comparable (both at -1.78 logits) . This is because 
those who can complete the first step of item 4 will tend to find 
the ensuing step relatively rasy (the first step is at a relatively 
higher threshold than the escond step). In the sample, none of the 
pupils at level 4 makes this Key type of error after successfully 
j^rouping the constant and unknown terms. The same occurs for pupils 
at performance level I because they cannot even complete the first 
step which for them is relatively difficult. This error pattern of 
the high and low ability pupils is consistent with the relatively 
higher step difficulty of the first step of grouping terms than the 
second step of simplification and evaluation. 

Item 5 (scale performance level between 2 and 3): 
Solve the equation 4(xf2)+2x«4. 

Nearly 52% of the pupils solve this equation correctly (01, 25%, 
69%, 91% respectively at the four levels of performance). Amongst 
the operations of removing brackets, grouping for conf^tant and 
unknown terms, and solving for the unknown, two key errors stem out 
that are unique to the given equation. The first is that while they 
have no difficulty in understanding "find the value of thoy 
cannot understand the technical word "solve". As a result, pupils 
stop at the first two stpps of removing brackets and simplifying. 




20%, 9%, and 10% of pupils at performance levele 1, 2 and 3 Mke 
this error. The second error stems from the uneasiness of "x** 
taking the value of a negative number. Pupils simply ignore the 
negative sign in the solution. 11% and 9% of pupils at levels 2 and 
3 commit this error. 

These two types of errors are characteristics of pupils of level 
3 and have nearly accounted for alT. the errors made by pupils at 
this level. Also, it is observed that these two types of errors are 
seldom made by pupils of performance level 4 and l. Clearly, pupils 
at performance level 1 cannot even remove the brackets to solve for 
the unknown. The response thresholds of the three steps of removing 
brackets, grouping and simplifying, and solving are 0.17, -0.50, 
and -0.20 loqits respectively. The two key errors occurred at the 
second and third steps respectively and these are of higher step 
difficulties than those key errors of items 3 and 4. 

Item 8 (scale performance level between 3 and 4) : 

Mangoes cost m c nts each and papayas cost p cents each. If I buy 

5 mangoes and 2 papayas, what does 5m+2p stand for? 

Nearly 57% of the pupils have no problem of understanding the 
algebraic expression correctly (0%, 30%, 71% and 91% respectively 
at the four levels ot performance) • The key error is that pupils 
misinterpreted **bm*' as 5 mangoes and "2p" as 2 papayas. Thus, 
"5m+2p** is interpreted qualitatively as the number of mangoes and 
papayas bought, 40%, 23%, 9%, and 9% of pupils of performance 
levels 1 to 4 make this error. 
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Furthrtmo!^, a related type of prior is that and 271 of 

pupils at performance levels 1 and 2 do not respond to thip. 
question, showing that they have difficulty in comprehend inq the 
given expression. Once the first step of underRtanding what "5m'* 
and "2p" mean the second step of stating that the expression 
stands for the cost of fruits is relatively stralghtforvard. The 
response thresholds of these two steps are 2,30 and -1»67 loglts 
respectively. The high response threshold of the first step of 
understanding clearly leads to an expectation that there is a 
higher proportion of pupils at performance levels 1 and 2 to treat 
variables "ro" and "p'* as objects or to omit the question entirely. 

Item 10 (scale performance level 4) : 

A mother promised to pay her son 20 cents for every math problem 
that he got right and fined him 5 cents for each wrong answer* 
After 10 problems, the mother had to pay the boy $1.50. How many 
problems did the boy answer correctly? 

Only 31% of the pupils can solve this algebra word problem 
adequately (20%, 18%, 31%, and 82% respectively at the four levels 
of performance). Two methods of problem solving are evident, 
namely, the algebraic metnod and the trial and error method. The 
latter kind is four times more popular for those successful problem 
solvers (G% and 25% comprising the 31% of successful problem 
solvers). There are comparable chances for successful problem 
solvers at performance level 4 to deploy the algobraic method (56%) 
than the trial and error method (44%) . However, successful problem 
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solvers at levels 2 and 3 cannot use algebraic method but need to 
use trial and error for problem solution (0% and 141 compared to 
100% and 86% at levels 2 and 3 for the two methods) . 

Consequently, pupils at intermediate performance levels can 
still use heuristics to search for a viable solution whereas those 
high-ability pupils can make use of the "deep structure" of the 
question and its associated knowledge base to attempt a solution 
in an algebraic way. This assertion is further supported by the 
greater percentages of pupils of performance levels 1, 2 and 3 
using only part of the question and superficial cues foi* their 
problem solving (40%, 25%i 33% and 9% respectively at the four 
levels of performance] . 

Another type of error is made by pupils of performance levels 
2 and 3 only (11% and 33% of pupils at the two levels 
respectively) . It occurs because although pupils understand the 
problem correctly and have the know-how to solve for a solution 
they make a mistake in representing the problem using the wrong 
algebraic equation. Pupils at performance level 1 do not commit 
this error because they cannot understand the problem at all 
whereas pupils at performance \evel 4 find the problem-solving 
procedures straightforward once they have understood the problem. 
These explain the inordinal pattern of the response thresholds of 
the four steps of problem-understanding, problem-representation, 
and the two steps of solving of the system of two simultaneous 
equations of unknown "x" and "y" (2.82, -1.04, 2.95 and 1,26 logits 
respectively). The third step is noticeably more difficult because 
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it involves solving equations of two unknowns. 

The findings of the error analyses on the relationships between 
key types of error and problem-solving ability are summarised 
below: 

1. For pupils at all the four levels, particularly the first three 
levels^ they rely on superficial cues when they do not understand 
a problem. In particular^ pupils at levels 2 and 3 will proceed to 
attempt a solution when some superficial cues can get them going* 

2. For pupils at level 4, they can use algebraic methods to 
solve word problems* whereas for pupils at levels 3 and 4 they can 
make use of other heuristics such as trial and error if they have 
understood the problem but fail to proceed using the algebraic 
method. Pupils at levels 1 and 2 treat variables as concrete 
objects so that they usually have difficulties in 
problem^understanding and problem-representation in an algebraic 
way. Pupils at these two levels very often cannot even proceed to 
the problem-execution phase of using the various algebraic skills 
and they use heuristics sparingly, 

3* For pupils at levels 2 and 3* they show a confusion between 
terminologies such as "solve" and "simplify" but do not have 
difficulties when a simple phrase such as "find the value of" has 
been used instead. Tliey tend to make errors in simplifying and 
evaluating algebraic expressions and cannot admit variables which 
take on negative values. The solving of simultaneous equations 
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involving two unknowns is also well beyond their abilities. 

Relationships between key types of errors sad problea-solvl&g 
sbilltf 

This section attempts to model statistically the key types of 
errors of the five selected items along some structural dimensions 
that are parallel to that of the problem-solving proficiency 
continuum. Latent trait modelling such as the Partial Credit 
Modelling cannot be used because of the unrealistic assumption that 
these key types of errors can be aligned onto a unidimensional 
trait. Instead, analytical toolr that do not rely on this 
unidimensional ity assumption are needed. Furthermore, it should be 
capable of modelling frequency counts of key types of errors and 
successful completion of items as those in Table 4.1. Dual scaling 
(Nishisato, 1980) is a versatile method for the exploration of the 
hidden structure of categorical data such as two-way contingency 
tables and quantification of row and column categories along 
parsimonious structural dimensions. Statistically, it decomposes 
a table into row and column structural dimensions, with optimal 
weights assigned to row and column categories along these 
diuensions, such that both the between-row and between-column 
discriminations arc simultaneously maximised. 

These discrimination indices are expressed by the "squared 
correlation ratio" following Guttman's principle of internal 
consistency (Guttman , 1941) in minimising within-subject (row or 
column) discrepancies and maximising between-subject (row or 
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column) differences in values of optimal weights. The method 
resenxbles a multidimensional decomposition of data with the most 
••informative" structural dimension extracted first, then the second 
most ••inf'ormative'* dimension, and so on, until the information in 
the data are exhaustively extracted. From the point of view of dual 
scaling, when the two dimensions of categories spanning the 
contingency table are independent of each other, there exists no 
structural dimension in the data because the information level of 
this table is zero. Consequently, associated with each structural 
dimension is a statistic "Delta Partial", indicating the 
percentage of information explained by that dimension. The optimal 
w^^ghts are further weighted to reflect this relative importance 
of structural dimensions for comparing amongst structural 
dimensions. These weighted optimal weights of a structural 
dimension can be plotted against another revealing the underlying 
structure of the data table. 

Table 4.1 provides all the necessary input for dual scaling. The 
two dimensions of this contingency table are firstly the four scale 
perfo^Tiance levels, and secondly the key types of errors together 
with the categories of counts on the successful completion of the 
five selected items along the proficiency continuum. Three 
structural dimensions result from the dual analysis, accounting for 
12%, 21% and 7% of the information in the data. The weighted 
optimal weights of the categories of the two dimensions of Table 
4.1 can be plotted along the first and second structural 
dimensions, showing both the hierarchical and dimensional 
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properties of thi^ k*-/ types of en or, ntaqrs oi pprformnncp levels, 
and ability levels in the successful completion of the? test items. 
This plot is shown in Figure 4.X. 

The first structurr%l dimonnlon clearly is the proficiency 
continuum upon which key types of errors are structured. 
Examinations of the coordinates J, 1. D, F, N, M, G, B relative to 
coordinates 1, 2, 3, 4 recapitulate exactly tho findings on how 
pupils of different performance levels may have the opportunities 
to commit tho various key types of errors. It demonstrates that by 
makinq reference to the firct structural dimension, those 
"inabil itirs" as defined by the key types of errors can be 
hierarchically structured in the came way as the proqression of 
knowinq/procedural skills in solvinq alqobtaic word problems. This 
is revealed by coordinates A, C, H, K that thoy are S'^quencod 
alonq the scale performance continuum by outcome. It sljoald be 
noted tr.at only if a pupil is well past performince level ? will 
s/he Ruccensfully complete questions 5, 4, S and B. This is in 
accordance with the probability of pupils of the four performance 
levels par.sinq the respective items. Also, in order to successfully 
solve the alqebra word problem (item 10), a pupil nerd to be in 
performant^e level 4. 

Furthermore, a pupil needs to be able to understand and represent 
a problem, as suqqested by the coordinates K, 4, J and 1 
interpreted alonq the second minor structural dimension. The 
position of I. nhows that it is mostly those level 3 pupils, who 
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possess most of the basic algebraic skills, when faced with 
problems in representinq the problem situations alqebraical ly will 
deploy tha trial and error methods. The first structural dimension 
thus constitutes the right arm of the competence ladder which is 
to be paired with the left arm of the problem-solving proficiency 
continuum described in the last chapter. The four performance 
levels act symbolically as the rungs linking the progression of the 
problem-solving steps on one arm, to the hierarchical structure -f 
key types of error on the other. This ladder shows a genuine 
realisation of quantitative assessment of problem-solving t-.sks 
with meaningful qualitative interpretations on both the structure 
of problem- solving steps and key types of errors that are firmly 
rooted ir the literature described earlier. 

A metacognitlva •Jtploration of the problem and saarcb spaca* of 
problem aolvers 

spice does not allow a detailed reporting here of how metacognitive 
decision making and attitudinal behaviours affect pupils engaged 
in problem solving. Instead, four excerpts from two pupils (Pi and 
r2) solving two algebra word problems (A and B) in front of the 
researcher (T) are discussed. These excerpts illustrate the 
contrasting problem and search spaces of the problem solvers. 
Problem A is a standard textbook question asking for the number of 
cows and chickens given the total number of the stock and the total 
number of legs. Both pupils do not have problems in understanding 
the question, knowing that cows have 4 legs whereas chickens have 
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2 legd* 

Pupil 1 simply does not have the slightest incentive to attempt 
an answer when he cannot recognise a to proceed. Pupil 2 

plunges* into solving the problem quickly by considering only the 
total number of legs. Hie method is faulty, leading him to an 
incorrect answer. He is not aware that his answer on the total 
number of animals is different from that given in the question. 
This shows that at this stage he is still using partial information 
in order to proceed with a solution. However, he recognises that 
he has not reached the goal yet because he does not know the number 
of cows. He does not Know what has gone wrong and is looking for 
an alternative method. He remembers that if he can write two 
equations to represent the problem he may solve the problem in an 
algebraic way. Clearly, pupil 2 is equipped with a domain-specific 
knowledge base for him to categorise some familiar problems and to 
solve the simultaneous equations of two unknowns. Despite his 
success, he fails to evaluate why his initial attempt has not 
worked • 



Excerpt Al; 

PI: A woman has altogether 50 cows and chickens. These animals 
have a total of 172 legs. How many cows are there? 

T : Do you und**rstand what the question is asking you? 

Pl: [nods her head) Yes. 

T : How will you answer this question? 
(pause] 
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What is the question asking you? 
Pi: How many cows are there? 
T : How will you solve it? 
Pi: [pause a while] 

I only know cows have 4 legs and chickens have 2 legs. 
T : What else do you know? 
PI: I don't know how to do it. 
T : You said you understand the question ... 

PI: Yes, I know what it's asking, hmm but I don't knc ^ how 

to find the answer. 
T : You have no idea at all? 
PI: No. 

Excerpt A2: 

P2: A woman has altogether 50 cows and chickens. These animals 

have a total of 172 legs, nnu many cows are thpre?. 
T : Do you understand what the question is asking you? 
P2: It just says that a woman has altogether 50 cows and chickens. 

Those animtils have a total of 172 leqs. 

A cow has 4 logs, chicken 2 logs, th«-\t is 6 legs. 

I think 172 divide by 6 that's 27 animals. 

But I'm not sure how many cows and how many chickens yet. 
T : why do you divide 172 by 6? 
r2: The total number of legs of the animals. 
T : You obtain 27 as the answer ... 
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P2: Yes, but I still do not know how many cows and chickens are 
there. 

May be I'll represent x for the cows and y for the chickens 

So > plus y equals 50, 

Then 4 plus 2 equals ... [pause] 
T : What are you trying to do now? 
P2: I'm trying to write another equation. 
T : So you're trying to use another method now. 

Why don't you want to use the first method? 
P2: May be that is the same as this one. 

May be I can get the answer here also. 
T : Okay, how would you procer i? 
P2: X is cow so 4x plus 2y equals 172. 

I'll substitute this as one [pointing to the first equation : 

xfy^SO) and that as two (referring to the equation : 

4x+2y=172] . 

[proceeded with solving a set of simultaneous equations]. 
I get X as 36 . . . , so there are 36 cows. 

Problem B is also thoroughly understood by both pupils, although 
the pupils in the sample may not have encountered similar questions 
before, this time, pupil i attempts an answer by making use of all 
pieces of given information in the question, since he lacks 
algebraic skills, he can only proceed in an arithmetical way. He 
starts by assuming that the son gets the 10 problems right, and 
ultimately through some more sensible steps, he obtains a 
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contradictory result that the son gets the 10 problems wrong* He 
is aware of this discouraging result and then gives himself up 
after he fails to see another way to proceed* 

Pupil 2 cannot obtain an answer this time because problem B is 
less familiar to him* He attempts an answer in an arithmetical way 
first and can go no further* The trouble is that the question is 
so well -designed that pupils will realise whether they are wrong 
or not. Realising that he is wrong, pupil 2 attempts to solve the 
problem in an algebraic way but in vain* He is confused but 
perseveres because he V^ows that if he can write two equations he 
can obtain the answer He allows hSnself a final attempt by 
rereading the question and studies soiae special cases, hoping by 
using trial and error he may see a structure in the proDlem to 
obtain an answer* All these attempts are fruitless and he has to 
admit that he cannot answer* 

Excerpt Bl: 

PI: A mother promised to pay her son 20 cents for every math 
problem that he got right and fined him 5 cents for each 
wrong answer* After 10 problems, the mother had to pay the 
boy one-*£if ty*How many problems did the boy answer correctly? 

T : Do you understand the question? 

PI : Yes . 

(teacher checks whether P understands the problem] 
T : Do you know how to do it? 

PI: 20 cents ..*, okay, 20 cents times 10 is 2 dollars* 
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Not all the 10 problem9 he got right. 
5 cents • • • , 

2 dollars minus one-fifty is 50 cents 

That neans 50 cents is his wrong answer* 

So how many 5 cents will there be if you want 50 

cents . . . , 

[counting with her fingers] 10 
I think it's wrong already* 
T : Why do you think it's wrong? 

PI: Because he cannot answer all 10 problems wrongly. 
T : Can you think of any other way to solve it? 
PI: No, I cannot do it. 

T : Have you seen such a problem before? 
PI: No. 



Excerpt B2: 

P2: A mother promised to pay her son 20 cents for every ir.ith 
problem that he got right and fined him 5 cents for each 
wrong answer. After lo problems, the mother had to pay the 
boy one-fifty. How many problems did the boy answer 
correctly? 

T : Do you understand the question? 

P2: Yes. 

T : Do you know how to solve it? 
P2: I'll try. 
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If ha got right one problem, he'll get 20 cents* 
so 1 problem 20 cents there are 10 probl< ms* 

10 times 20 is 2 dollars 

If he get 10 problems right, he'll get 2 dollars. 

If he get 10 sums wrong, 5 cents times 10 50 cents 

he has to pay 50 cents* 

(pause] 

T : What are you thinking now? 
P2: I'm trying to get an equation hmmm 
T : What sort of equation are you trying to get? 
P2: Something plus something equals one-fifty, 
[pause] 

T : What are you thinking now? 
P2: I'm confused. 

T i But you understand the question? 

P2: Yes, but I cannot set up an equation because of the 

20 cents and the 5 cents* 

Let me see if he get 5 sums right hmm 

that will be one dollar. 

Then the 50 cents I'll add two more sums to make it 

one-forty [scratch her head and giggles] 

I think I did it wrongly. 
T : why do you think so? 
P2: I think I should get an equation 

(rereads the question]* 

5 sums one dollar, two more sums one-forty 
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but this is not algebra* 
T : You'll prefer to use the algebra method? 
P2: I think it»s easier, but I can't set the equation. 
T : Do you want to proceed with this method? 
P2: I don't think it's correct* 
T : Do you want to try further? 
P2: No. I don't think I can get the answer* 

Towaras a pedagogy of problra solving on algebra 

This section presents some findings that contribute to a pedagogy 
of solving algebra word problems* Some useful questions for helping 
pupils monitor their problem solving are also given. Three lines 
of research findings are evident: 

1. In order to solve an algebra word problem successfully, pupils 
need to be conversant with a number of exemplary problems so that 
problem-categorisation and problem-representation can be made 
easier. Otherwise, even though pupils do not have any difficulties 
in understanding the given problem, they cannot represent it in a 
way for an algebraic solution. Those who are keen may use a trial 
and error method or approach the problem in an arithmetical way. 
It should be noted that problem-categorisation and 
problem-representation cannot be completed if the surface (eatures 
have not been sorted out by the pupils. The reason is that puf ils 
may not recognise the underlying "deep structures" that are meant 
for them. These two steps of categorisation and representation are 
vital for solving word problems and piove most difficult even for 
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the high ability pupils. 



2. Pupils plunge into their solving of the problems often too 
quickly, failing to recognise that there are important cues that 
have to be used. As a result, pupils often cannot represent a 
problem properly, when reaching an impasse, they also lack 
heuristic strategies to explore the problem in a systematic way* 
It should be noted that associated with each problem representation 
there is a domain-specific knowledge base. In solving algebra 
problems, pupils are thus keen to find equations linking the givens 
to the unknowns. Solving a variety of problems which have the same 
"deep structure" in numerous contexts may help pupils in the 
setting up of these equations. Preferably, the problems should be 
designed in such a way that pupils will immediately recognise 
whether their solution paths will lead to a detour or blind alley. 

3. There is a hierarchy of concepts and procedural skills showing 
how pupils of different competencies progress towards mastery. In 
^ parallel manner, pupils of different levels of abilities commit 
different types of errors. Consequently, instructions may be 
applied according to pupils' performance levels. The fashionable 
constructivist teaching approaches in the educational literature 
will help pupils construct their concepts. As far as procedural 
skills are concerned, rehearsal within a personally meaningful 
context may drive mastery towards automation. Engaging pupils in 
problem solving tasks reveal the diversity of perceptions and 
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approach , so much needed to be understood by clcissroom teachers 
if meaningful learning is the outcome of schooling* 

As far as metacognition is concernedr the following questions 
may be asked covertly or overtly when pupils are engaged in problem 
solving* These questions aid one's awareness of the progi w 
towards a goal state, and at the same time control and monitor the 
solution path in the problen-solving process. These questions are 
relevant to the various stages of problem solving, namely, 
problem-understanding^ problem-representation, problem- execution, 
problem-control, and problem-evaluation* 

• What the problem asks ne to do ? 

• Do I need to break a problem into subproblems ? 

• What qoa.l -directed content knowledge and heuristic strategies 
are acceaaible to me ? 

• Is there a related problem or pattern seen before ? 

• Do I need to look at some simple cases for some tentativr 
explorations ? 

• Do I need to make a quess and check ? 

. Is the problem representation appropriate and complete ? 

. When to apply particular heuristic strategies ? 

. When to pursue alternative solution routes ? 

. When to review progress, within or between episodes, on how 

things are and where they lead to ? 
. What to do when an impasse has been reached ? 

. Am 1 surprised, irritated, frustrated, anxious, and confused ? 
. can I tolerate ambiguity of results and premature closure ? 
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. Is th€*re another way to look at the problem ? 
. Do I need to check solution by retracing steps ? 

In sum, the present chapter completes the discussion on the 
meaningful assessment of problem-solving tasks in the classroom* 
The problem-solving network, the proficiency continuum, the 
hierarchical structural dimensions of errors, the problem and 
search spaces of problem solvers, and most important of all, the 
conceptual framework which is based on a theory of perception logic 
and construct ivist information processing all contribute to the 
proposed art and technology of meaningful asssessment of 
problem-solving tasks in the classroom. 
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Appendix 1 



ALGEBRA TEST 



Time: 60 minutes 



1« Simplify 2a+5bM. 

2. Simplify (8y4^6z) - (4y+3z) . 

3. What is the value of 3a-b+5c if a»2, b«3 and c=57 

4. Given that 4y-8 - 2y-4, find the value of y. 

5. Solve the equation 4{x+2) + 2x « 4. 

6. One stick is twice as long as the other. If the length of the 
shorter one is x cm, find the length of the longer one. 

7. The price of a desk and a chair. is $x. What is the price of 
the desk if the price of the chair is: i) $20, ii)Sy? 

8. Mangoes cost m cents each and papayas cost p conts each. J* I 
buy b mangoes and 2 papayas, what does 5m ^ 2p stand for? 

9. Mary's basic wage is S500 per month. She is also paid another 
$2 for each hour of overtime that she works. If h stands for 
the number of hours of overtime that she works and if w stands 
for her total wage in $, 

i) write down an equation connecting v and h, 

ii) wha*- '"^uld Mary's total wage be if she worked 10 hours of 



10 A mother promised to pay her son 20 cents for every Kaths 

problem that he got right and fined him 5 cents fcr each wrong 
ansv;or. After 10 problems, the mother had to pay the boy 
$1.50. How many problems did the boy answer correctly? 

\l A woman has altogether 50 cows and chicken??. These animals 
have a total of 172 legs. How many cows are there? 
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